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Abstract 

Simulation and visualization of rover be- 
havior are critical capabilities for scientists 
and rover operators to construct, test, and 
validate plans for commanding a remote 
rover. The VIPER system links these capa- 
bilities, using a high-fidelity virtual-reality 
(VR) environment, a kinematically accu- 
rate simulator, and a flexible plan execu- 
tive to allow users to simulate and visualize 

possible executiorroutcomesxif^plairunder 

development. 

This work is part of a larger vision of a 
science-centered rover control environment, 
where a scientist may inspect and explore 
the environment via VR tools, specify sci- 
ence goals, and visualize the expected and 
actual behavior of the remote rover. 

The VIPER system is constructed from 
three generic systems, linked together via a 
minimal amount of customization into the 
integrated system. The complete system 
points out the power of combining plan ex- 
ecution, simulation, and visualization for 
envisioning rover behavior; it also demon- 
strates the utility of developing generic 
technologies, which can be combined in 
novel and useful ways. 

1 Introduction 

Imagine trying to drive when your vehicle only does 
approximately what you command, you only catch 
occasional glimpse of your environment, 20 minutes 
pass between your command and the vehicle’s re- 
sponse, and to top it off, you don’t really know how 
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your vehicle works. This is the world of scientist- 
directed planetary rover exploration. 

Planetary rovers are scientific tools for exploring 
an unknown world. One focus of the Autonomy and 
Robotics Area (ARA) at the NASA Ames Research 
Center is to design and develop the tools and tech- 
niques that allow scientists to control a rover effi- 
ciently and effectively. This presents challenges both 
in the user interface and in the underlying rover con- 
trol methods. 

One important element of the planetary rover con- 
trol is the ability to simulate and visualize possible 
execution outcomes of a plan under development. 

— We-ha v e d eveloped the VIPER system, -which links 
plan execution, rover simulation, and a high-fidelity, 
realistic virtual-reality (VR) environment. This sys- 
tem is one part of a larger architectural design under 
development that includes tools for science goal spec- 
ification and plan generation. 

The ultimate vision for the overall architecture is 
that scientists at “mission control. “ and potentially 
elsewhere in the world, will both specify and ob- 
serve the rovers operation as well as science prod- 
ucts through the VR environment. The scientists can 
examine physical features of the environment (dis- 
tance, volume, cross-sections) and specify science- 
level goals, for example to go to a rock and drill a 
small sample. These goals are then interactively re- 
fined at mission control with the help of a planning 
and scheduling system, adding constraints of rover 
motion, resources, and time to arrive at a final plan. 
Once the plan is ready, it is communicated to the 
rover. 

On board the rover, the plan is executed by testing 
and monitoring conditions on time, resources, and 
rover and environmental state. The same plan ex- 
ecuted multiple times may produce many different 
behaviors, based on the initial conditions and the 
variability of the rover’s interactions with the envi- 
ronment. 

The VIPER system allows users to simulate and 
visualize possible execution outcomes of a plan un- 
der development. We have developed a kinematically 
accurate simulator of the rover that allows the sci- 
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Figure 2. Overview of VIPER plan execution, simulation and visualization. 


to MarsMap. initially deployed during the Mars Po- 
lar Lander mission [Nguyen et a/., 2001 J. Viz im- 
plements an architecture that allows a flexibility 
and customizability similar in spirit to VEVI and 
presents ffie user with a highly interactive imrner- ~ 
sive environment as in MarsMap. 


Robot Behavior Simulation The behavior of 
the mechanism is reproduced by the VirtualRobot 
simulator. VirtualRobot was initially developed at 
the Swiss Federal Institute of Technology, Lausanne 
(EPFL) by the Virtual Reality and Active Interfaces 
Group (VRAI) as an interactive tool to control and 
study any kind of robot manipulator [Fluckiger et o/. t 
1998, Fluckiger, 1998]. VirtualRobot was based on 
a generic kinematic generator. The collaboration of 
the VRAI Group with the Autonomy and Robotics 
Area of NASA Ames led to extensions of Virtual- 
Robot that enable the simulation of rovers in addi- 
tion to robot manipulators. 


Plan Execution The plan execution component 
of this work was inspired in part by work on the Re- 
mote Agent (RA), an integrated agent architecture 
developed for spacecraft control and deployed as an 
experiment on the Deep Space One mission [Bernard 
et nL, 1998; Muscettola et a/., 1998]. The rover exec- 
utive demonstrates advances in conditional execution 
compared to the RA executive: the language of the 
RA. executive does not accept conditional sequences, 
which are critical to the success and effectiveness of 
a rover mission, given the highly variable interac- 
tions of the rover and the environment. The current 
implementation of the rover executive does not, how- 


ever, attempt to reproduce all of the capabilities of 
the RA; in particular, multiple concurrent activities, 
model-based state reconfiguration, and run-time re- 
source selectio n for sta te variables are not currently 
included in our executive. 

VIPER system organization The VIPER sys- 
tem comprises the plan execution, simulation, and 
visualization subsystems (see Figure 2). These com- 
ponents allow the scientists to explore different pos- 
sible plans and the expected behavior of the robot in 
the virtual environment. 

The plan execution component interprets the com- 
mand plan, checking conditions and monitoring run- 
time requirements of the plan. It sends commands to 
the rover simulation component and receives state in- 
formation back. The rover simulation component, in 
turn, simulates the kinematics of the rover and its in- 
teractions with the terrain. The simulator sends pose 
information to the visualization component, which 
continually updates its environment model and ren- 
ders the scene for the viewer. 

3 Underlying Technologies 

The VIPER system is built on generic technolo- 
gies for each of its subsystems, which are special- 
ized with data or configuration information to work 
with the particular robotic platform and environ- 
ment. This allows the system to be used for visual- 
ization, test, and design of different, novel, and even 
imaginary robotic platforms. In particular, parts of 
this technology have been used to model and sim- 
ulate the Pathfinder environment, the Mars Polar 
Lander robotic arm and camera, the NASA Ames 
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Figure 4: Simulation of K9 rover driving over a rock. 

existing science toolset interface providing manual 
control of the rover and its image sensors — this is 
used for sequence construction. In addition, a but- 
ton was added to launch the VirtualRobot simulator. 
The total effort required was a few days. 

3.2 VirtualRobot Simulation 
VirtualRobot was initially developed as an interac- 
tive tool to control and study any kind of robot ma- 
nipulator (Fliickiger et a/., 1998; Fliickiger, 1998]. 
The design of VirtualRobot was driven by the fol- 
lowing requirements: 

• no code writing~nor kinematic model is necus-' 
sary to describe the kinematic behavior of the 
robot, 

• the program is able to handle the kinematics of 
any robot manipulator structure in real-time 1 , 

• an intuitive user interface allows both novices 
and experts to easily manipulate robots in a vir- 
tual environment. 

For these reasons VirtualRobot was based on a 
generic kinematic generator: after reading a text file 
describing the geometry of the robot, the program 
builds a numerical solver which computes the di- 
rect and inverse kinematics of the robot. The robot 
structure can be serial (as most industrial robots), 
parallel (as a Stewart platform) or hybrid (mix of 
the previous two). A robot model is created in a 
virtual environment and the user is able to interact 
with the robot using intuitive 3D input devices (like a 
Space-Mouse) or 6 degree-of- freedom force- feedback 
devices. 

VirtualRobot has been extended to enable the sim- 
ulation of rovers in addition to robot manipulators. 
The same kinematic solver can now also accommo- 
date multi-wheeled rovers with passive suspensions 
driving on uneven terrains (see Figure 4). In addi- 
tion, rather than being controlled by user input, the 
kinematic solver responds to inputs coming from any 
other program through the network. 

l wo consider real-time from a human point of view: 
refresh rate in the range of 20Hz to 200Hz 
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Figure 5: Excerpts oFa robot hie description used by 
VirtualRobot to build and simulate a rover 

The following two sections briefly review the robot 
description file and the kinematic solver used in Vir- 
tualRobot. 

Robot description file 

Any robot manipulator, rover or combination of both 
can be described in a human readable text file which 
is parsed by VirtualRobot. This file contains all the 
geometric properties of the mechanical structure to 
be simulated. The robot structure can be expressed 
with the Khalil-Kleinfinger formalism [Khalil and 
Kleinfinger, 1986] (which is also usable for multi- 
branch structures, unlike the well known Deaavit- 
Hartenberg [Denavit and Hartenberg, 1955] nota- 
tion) or by using regular reference frames (3 transla- 
tions + 3 rotations). The robot is defined as a tree 
structure from the base up to each end-effector or 
wheel. For robots with kinematics loops, the desired 
chain is closed by defining am additional constraint 
between two branches of the tree. Similarly, declar- 
ing a constraint between any body of the structure 
and an input (3D device, network, etc.) will make 
the VirtualRobot program compute the appropriate 
inverse kinematic behavior of the structure to satisfy 
these constraints. 

The use of a generic description file (see Figure 5 
for an example), to define robots allows rapid sim- 
ulation creation for new robot and rover structures 
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Figure 6: Excerpts of a CRL plan. 


tions are respected. CX can be instructed to wait 
for a subset of the preconditions (for example, the 
start time window) rather than failing the execu- 
tion of the node. CX receives state information from 
the low-level rover control (or simulation). In the 
envisioned overall rover architecture. CX will also 
receive higher-level state information from a state- 
identification module and resource information from 
a resource manager. It uses this information to check 
preconditions and maintenance conditions, as well as 
to check the eligibility conditions of the plans in the 
alternate plan library. 

At each point in time, CX may have multiple 
options, corresponding to the eligible options of a 
branch point and the enabled alternate plans. CX 
chooses the option with the highest estimated ex- 
pected utility, computed over the remainder of the 
plan. In the current implementation, the utility of 
successfully completing an atomic action is fixed and 
set by operators at mission control. From this atomic 
utility and a model of the probabilities of various 
events (such as a traverse taking longer than antic- 
ipated), the expected utility of an entire branching 
plan can be calculated. 

When plan execution fails. CX reacts as specified 
in the node, either ignoring the action or aborting 
the execution and checking for applicable alternate 
plans. If execution is aborted and no alternate plans 
apply. CX aborts the entire plan set and puts the 
rover into a stable standby mode; all operation is 


suspended and the rover awaits further plans from 
mission control. 

CX and CRL were incorporated into the rover au- 
tonomy architecture used to control the Marsokhod 
rover during a February 11)99 field test (Bresina et 
aL, 1999) and the K9 rover during a May 2000 field 
test [Bresina *t a/., 2001]. 

Customization of CX and CRL for VIPER 
The CRL grammar is generic; only the command 
and condition names change from one application to 
another. The CX*execution semantics rely on the 
general CRL properties and not the command and 
condition names. As such, the central “execution en- 
gine is completely generic. The only specializations 
needed are in terms of communication with the exter- 
nally controlled rover (or simulation). These are ac- 
complished with C-h-f- subclassing that is completely 
transparent to the execution engine. 

In order to control the K9 rover, the CRL “com- 
mand dictionary" was defined, as well as routines 
that CX calls to translate CRL commands to mes- 
sages to the rover. To incorporate CX and CRL into 
VIPER, the same command dictionary was used, but 
the translation routines were changed to communi- 
cate with the simulation rather than the actual rover. 

In all, the total effort was no more than a week, much 
of that to separate out the parts common to the ac- 
— . tua l an d simul ated rover to avoid code^ duplication. 

4 Illustrative Example 

Consider a simulation of a rover moving in the Mars 
Pathfinder environment. VIPER presents the viewer 
with three main windows that show; I) a 3D visu- 
alization of a mobile robot at the Mars Pathfinder 
landing site executing a plan, 2) a display showing 
the VirtualRobot parameters, and 3) a 2D text dis- 
play of the robot executive status. See Figure 2 for 
representative screen images. 

4.1 Scenarios 

Consider the scenario where the rover is located next 
to the lander, as in Figure 7. The next goals to 
achieve may include getting a close-up mosaic of 
Yogi, the largest rock in the environment, followed by 
a traverse around the lander to near the second ramp 
(Ramp2). Depending on the data storage available 
after the mosaic ends, the rover may decide to tra- 
verse via either side of the lander; one has more rocks 
that are interesting scientifically, the other has fewer. 

In either case images will be acquired of the targets 
during the traverse. If the time is too short, the rover 
will remain near the first lander ramp (Rampl). 

These three main scenarios, shown schematically 
in Figure 8 and graphically in Figure 7, are the result 
of data, power, and time limitations on plan execu- 
tion. 

The first scenario illustrates the effects of a time 
shortage in the absence of any data storage short- 
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Figure 9: Scope of MSF. 


Figure 10: MSF Sample Simulation Instantiation. 


time simplify their integration into a more reliable 
simulation. 

As shown in Figure 9. MSF will provide simulation 
environments which could be used at different level of 
— rntegratlO^rTrFoT'exaniple a research lab "could" have 
a complete system from the top level autonomy to 
the low level hardware control: in this case MSF will 
only provide a replacement for the robotic hardware 
and the environment with the simulation. On the 
other hand, one could need to test a very high level 
autonomy component, without having the rest of the 
system: in this case, MSF will also provide generic 
components replacing the missing parts. 

From an implementation point of view, MSF will 
rely on the publish/subscribe scheme used in HLA: 
each component of the simulation will communicate 
with the other components using a standard set of 
objects/ messages defined for the purpose of MSF. 
The HLA Run-Time-Infrastructure (RTI) manages 
all the communications between the participants of 
the simulation. The RTI also provides facilities to 
address the problem of Time- Management which is 
a key point in a simulation like MSF because its dif- 
ferent components are not necessarly designed to run 
in real time. 

The Figure 10 shows a simple simulation example: 
it is composed of several components which are all 
connected to the RTI for communication. They also 
have access to a separate database to avoid overload- 
ing the network when accessing large objects like im- 
ages. The autonomous software is decomposed into 
two distinct objects for clarity: each uses a differ- 
ent set of messages to interact with the simulation. 
Consider in this example that they send commands 
to a simulated robot. The kinematic simulator (like 


VirtualRobot) will then compute the behavior of the 
robot in response to the commands and return var- 
ious sensor updates. If the autonomous software re- 
quests the acq uisitio n of scientific data (like an im- 
age), then the sensor data generator will return ei- 
ther simulated data or real data extracted from a 
database. Visualization tools (like VIZ) could also 
be connected to the simulation to help the user eval- 
uate the behavior of the autonomous system. 

The development of MSF will provide a set of tools 
(simulation components / robotic systems library) 
that should speed up the testing process for the de- 
velopers of autonomous systems. In addition, the 
proposed framework will help advance the develop- 
ment of standardization of communication between 
autonomous software and robotic hardware. 
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